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1 Introduction. 

The object of this paper is the relation between the Szego kernel of an ample 
line bundle on a complex projective manifold, M, and the Szego kernel of 
the induced polarization on the quotient of M by the holomorphic action of 
a compact Lie group, G. 

Let M be an n- dimensional complex projective manifold and L an ample 
line bundle on it. Suppose a connected compact Lie group G acts on M as 
a group of holomorphic automorphisms, and that the action linearizes to L. 
Without loss of generality, we may then choose a G- invariant Kahler form Q 
on M representing C\(L). We may also assume given a G- invariant Hermitian 
metric h on L such that the unique Hermitian connection on L compatible 
with the holomorphic structure has normalized curvature — 2irifl. 

Let $ : M — > g*, where g is the Lie algebra of G, be the moment map 
for the action (this is essentially equivalent to the assignment of a lineariza- 
tion). Suppose that G g* lies in the image of $, that it is a regular value 
of $, and furthermore that G acts freely on $ _1 (0). In this situation, the 
Hilbert-Mumford quotient for the action of the complexification G on M 
is nonsingular, and may be naturally identified with the symplectic reduc- 
tion Mq = $ _1 (0)/(j. The latter, furthermore, inherits a naturally induced 
polarization Lq, with an Hermitian metric ho and a Kahler form Qq. The quo- 
tient structures Lq, h Q , Q are induced simply by descending the restrictions 
of L, h, n to fc-^O) down to M [HS2j . 

Let H°(M, L m ) and H°(M ,L^ k ) denote the spaces of holomorphic sec- 
tions of powers of L on M and of powers of Lq on Mq, respectively. Then 
G acts linearly on H°(M, L® k ), and in the given hypothesis by the theory of 
[GS2J there is for every integer k > a natural isomorphism H°(M, L® k ) G = 
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H°(M ,Lf k ). Here H°(M,L® k ) G C H°(M,L® k ) is the subspace of G- 
invariant holomorphic sections. 

The given choices equip the vector spaces H°(M,L® k ) and H°(M , L® k ) 
with the following unitary structures (, ) and (, )o, respectively. If a, r E 
H°(M,L® k ), then 

k 

M 



a,r) =: / hf k (a(p),r(p))vo\ M (p), 



where voIm = : Q An is the volume form on M associated to the Kahler form Q, 
and h® k denotes the induced Hermitian metric on L® k . Similarly, if <7o, To E 
H°(M ,Lf), then 

Oo,^o)o =: / h® k (a (p),T (p))vo\ Mt> {p), 

where now vo1a./ =: f2 x ^ n_9 ' ) is the volume form on M associated to the 
Kahler form Q . 

It is then natural to ask the following question: what are the asymptotic 
metric properties, with respect to these unitary structures, of the isomor- 
phisms r k : H°(M,L® k ) G -> H°(M ,L® k ) as k -> +00? To simplify the 
exposition, we shall mostly leave the isomorphisms implicit. From an 
analytic viewpoint, this question may be phrased as follows: how does the 
Szego kernel of the pair (M, L) relate to the Szego kernel of the quotient pair 
(M ,L )? 

A natural measure of the difference between the two unitary structures is 
offered by the comparison of orthonormal basis. In this direction, recall the 
following basic fact from the theory of algebro-geometric Szego kernels [Z] : 

given for every integer k > an orthonormal basis j of H°(M , L® k ), 

we have an asymptotic expansion, uniform in p 6 M , 



1>1 



where n is the complex dimension of M, g the real dimension of G, and 
the a/'s are smooth functions on Mo; the right hand side of Q is of course 

independent of the particular choice of the orthonormal basis j • In order 

to measure how far an orthonormal basis of H°(M,L® k ) G is from being an 
orthonormal basis of iJ°(M , LQ k ), we may then look for a similar asymptotic 
expansion for orthonormal basis of H°(M, L® k ) G , along ^(O) C M. 
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The following is a special case of Theorem 1 of [P] : Given for every integer 
k > an orthonormal basis |^ G '^| of H°(M, L® k ) G , we have an asymptotic 
expansion, uniform in p G $ _1 (0): 



j l>\ 



where the a^'s are smooth G-invariant functions on $ _1 (0), and a (p) > 
for every p G <& _1 (0). The same expression is rapidly decaying when 
p G" $ _1 (0). We are thus led to ask whether the r^'s are asymptotically 
conformally isometric, by a conformal factor involving an appropriate power 
of k. 

The following Theorem shows that this is not the case, unless the effective 
potential V e s of the action is a constant. Recall that V e s is the smooth 
function on Mq whose value at po G Mq is the volume of the fibre $ _1 (po) Q 
$ _1 (0) in the restricted metric. This is an important attribute of the action, 
playing a crucial role in the study of the Kahler structure of the quotient 
[BG . It may be viewed as the G- invariant function on $ _1 (0) associating to 
each p G $ -1 (0) the volume of the fibre G ■ p. 

Theorem 1. Suppose as above that G g* is a regular value of the moment 
map, and that G acts freely on $ _1 (0). Let n be the complex dimension of 

M and g be the real dimension of G. For every integer k > let |^ G '' C ^| be 

an orthonormal basis of the space H°(M,L® h ) G of G -invariant holomorphic 
sections of L® k . Then there is an asymptotic expansion, uniform in p G 

E \\ s ?' k) (P) 2 - v^pj ^ + £ a ^ kn ~ 9/2 ~ l - ( 3 ) 



i>i 



Let now the weights {u} label the finite dimensional irreducible represen- 
tations of the connected compact Lie group G; let be the representation 
associated to the weight u. Given the linear action of G on H°(M,L® h ), 
there is a G-equivariant direct sum decomposition 



<8ik\ 



H°(M,L® k ) = 0i/ o (M,L 

U! 

of H°(M,L® k ) over the finite dimensional irreducible representations of G; 
for every weight u the subspace H°(M, L® ) w is G-equivariantly isomorphic 
to a direct sum of copies of V u . Pairing Theorem ^ above with Theorem 1 of 
|P] . we immediately get: 
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Corollary 1. In the hypothesis of Theorem^ letu be the weight correspond- 
ing to a finite dimensional irreducible representation ofG. For every integer 
k > let j be an orthonormal basis of the subspace H°(M, L® k ) UJ C 

H°(M, L® ) . Then there is an asymptotic expansion, uniform inp G $ _1 (0), 

EN" ,fe V) 2 - ^^^ 9/2 + E^w^ /2 "- ( 4 ) 
i 11 Vcs( - p) i>i 

The asymptotic expansion in Theorem^ is a straightforward consequence 
of the following analytic core result, which we state here rather loosely and 
that will be spelled out more precisely in the course of the paper: 

Key Result: The Szego kernel of the triple (M, L, h) descends, in an appropri- 
ate sense, to an elliptic Toeplitz operator on the circle bundle of the quotient 
triple (M ,L ,h ), and after a suitable renormalization the symbol of this 
Toeplitz operator is (proportional to) the effective volume of the action. 

The techniques in this paper are based in a general sense on the microlocal 
theory of the Szego kernel |BSj . and more specifically on its formulation based 
on Fourier-Hermite distributions developed in |BGj : in this framework one 
has a good control of the functorial behaviour of Toeplitz operators and 
their symbols under geometric operations like restriction and push-forward. 
The general strategy used here was also inspired by the approach of Shiffman, 
Tate and Zelditch to Szego kernels on toric varieties jSTZj . especially by their 
philosophy of restricting the Szego kernel of an ambient projective space to 
a projective submanifold. In fact, here we shall first restrict the Szego kernel 
of (M, L, h) to the locus where the moment map vanishes, and then push it 
forward by the G-action. 



2 Metalinear and metaplectic preliminaries. 

We shall make use of the notions of metalinear manifold and half- form |GSlj . 
If Z is a manifold, we shall denote by |/\| (Z) the line bundle of densities on 
Z, and by |S7| (Z) its space of smooth global sections, which we simply call 

1 /2 

densities on Z. If Z is a metalinear manifold, we shall denote by f\ (Z) the 
line bundle of metalinear forms on Z, and by Vl}/ 2 (Z) its space of smooth 
global sections, which we simply call metalinear forms on Z. An orientation 
on a manifold induces a metalinear structure on it, and a volume form induces 
a nowhere vanishing half- form, its square root. If ^u, v 6 fi 1 / 2 (Z), the product 
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/j, ■ v is a density on X. The space of square integrable half-forms on an 
metalinear manifold has a natural Hilbert structure, given by 



< /!, v > = 




There is a more general notion of metalinear vector bundle, a metalinear 
manifold being a manifold with a metalinear tangent bundle. To a metali- 
near structure on a rank-r vector bundle E there is associated a line bundle 

1 /2 

f\ 1 [E] satisfying 

1/2 1/2 r 

/\(E)®/\(E) = /\(E*), 
where the latter is the top exterior power of the dual E*. The space of smooth 

1 /2 1 /2 

global sections of f\ (E) will be denoted by ft ^ . Thus, if Z is a metalinear 
manifold then /\ 1/2 (Z) = /\ l/2 (TZ) and QV 2 (Z) = fiJ/J. If E is oriented, it 
is metalinear, and ifO—>-A—>-E^B^Oisa: short exact sequence of vector 
bundles on a manifold, then any of the three vector bundles is metalinear if 

1 / 2 1 / 2 1 / 2 

the other two are, and f\ (E) = /\ 1 (A) ® /\ {B). A nowhere vanishing 
smooth section of f\ r (E*) induces a metalinear structure and a nowhere 
vanishing section of 2 , v/cx e f^ 2 . 

In the following, we shall be making various natural choices of a nowhere 
vanishing section of /\ r (E*) for some vector bundles in our picture (r be- 
ing the rank of E); we shall generally denote this choice by vol^ and the 
associated nowhere vanishing section of f\^^ 2 {E) by vo\^ 2 ^ = a/voIe. 

Let P and Q be metalinear manifolds. A morphism of metalinear ma- 
nifolds from P to Q is the assignment of a smooth map / : P — > Q and a 
morphism of vector bundles / : /* ( y /\ 1/2 {Z)^ -> A 1/2 ( p )- Let / : P -> Q 
be any smooth map. Let 

r. f = { ( (p, -{d P f)\ri)\ (/(p), 

be the conormal bundle to the graph of /. Then is clearly a Lagrangian 
submanifold of T*P x T*Q and a metalinear manifold. Furthermore, giving / 
the structure of a morphism of metalinear manifolds is equivalent to assigning 
an appropriate half- form on Tf |GSlj . 

Let us suppose, in particular, that Q is an oriented Riemannian manifold. 
We shall denote by t]q the half-form taking value one on oriented orthonormal 
frames of TQ, and call it the canonical half-form of Q. If P is also an 
oriented Riemannian manifold, we can then make any smooth map / : P — > 
Q into a morphism of metalinear manifolds by setting /(?7q) = f]p- Let 
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T/ C T*(P x Q) \ {0} be the conormal bundle to the graph of /, with 
projections qi : Tj — > P and g 2 : T/ — ► Q- Given the exact sequence 

— g 2 *(T*Q) - r(r 7 ) — ?1 *(TP) — 0, 

the half- form on associated to / is 

If a morphism of metalinear manifolds is given, passing to global sections 
we can define the pull-back of a metalinear half-form on Q to a metalinear 
half-form on P; we shall simply denote this by /* : Vt l / 2 (Q) — > Q}I 2 {P). If, 
in addition, / is a proper submersion, we can also define a push-forward op- 
eration /* : Q}/ 2 {P) — > fi 1 / 2 (Q). The microlocal theory for these operations 
on Fourier and Fourier-Hermite generalized half-forms has been developped 
in (HSU and |BHj . 

The proof of the following Lemma is left to the reader: 

Lemma 1. Let P, Q be oriented Riemannian manifolds, and let f : P — > 
Q be a proper submersion. Let us make f into a morphism of metalinear 
manifolds by setting /*(^q) =: rjp. For q G Q, let v(q) denote the volume of 
the fibre f~ 1 {q) in the induced metric structure. Then the associated push- 
forward operation /* : Vt l l 2 (P) — > Q}/ 2 (Q) satisfies 

f*M = v -vq- 



We shall now review some basic facts and notation from metaplectic ge- 
ometry. Let Sp(£) be the group of U x 2£ (real) symplectic matrices, and for 
1 < k < i define: 

Sp(k,£) = {A = [oij] G Sp(£) : ay = if 1 < j < k, i > k} . 
There is an obvious (surjective) morphism of Lie groups 

Sp(Jfe, £) GL(Jfe) x Sp(£ - k). (6) 
The following lemma is left to the reader: 
Lemma 2. We have: 

( (h B A n A 12 \ 
ker(w) = J ° \ k A } 2 A Q 22 :BG% i (t),4eM ¥ 4(l), 
[\0 -B l I t _ k J 

and An G M^M.), A 2 2 G Mg_fc(R) are both symmetric } . 
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Let T =: ker(ro) C Sp(k,£). Then T is a contractible, hence simply 
connected, normal subgroup of Sp(k,£). 
Similarly, let us define 



s P (M)- 



' (an 



Ofcl 





I V o 



det 



a-ik ^lfc+i 

Ofcfc O-kk+l 

a^fc+i 
• aik^ 
. : > 



aw \ 

&k+H 



e Sp(M) 



By restriction of zu, we obtain a surjective morphism of Lie groups, with 
kernel T C 3p(k,£) + : 



Sp + (/c, 



GL + (/c) x Sp(£ - k). 



(7) 



Therefore, Sp + (A;,£) is a connected subgroup of Sp(£). 

Let now s : Mp(£) — > Sp(£) be the metaplectic double cover. Now GL(£) 
sits naturally in Sp(£) in the standard manner, and 

Lemma 3. s- 1 {Gh(£)) =ML(f). 

Let us next define Mp(k,t) =: s' 1 (Sp(Jfe, £)), and Mp + (A;, £) =: s" 1 (Sp + (M))- 

Lemma 4. Mp + (k,£) is a connected Lie group. 

Proof. Let us consider the injective morphism of Lie groups ip : Sp(£ — k) — > 
Sp(£) given by 

/4 o 0\ 



Ai A 2 
A 3 A 4 



(8) 



Ai A 2 

0040 

\0 A 3 A 4 / 

Clearly, ^ (Sp(£ — fc)) C Sp(/c,£) + , and ip induces an isomorphism of homo- 
topy groups ip* : 7Ti(Sp(£ - fc)) ^ 7Ti(Sp(f)) ^ Z. Thus, 

s^ 1 [y>(sp(£ - fc))] = Mp(£ — k) 

is the unique connected 2:1 cover of Sp(£ — k). Hence, on the one hand, 
Mp(£ — k) is contained in the identity component of Mp(k,£) + , and on the 
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ker(7) 




other if there was another connected component of Mp(k,£) + then points in 
ip (Sp(£ — k)) would have more than two inverse images in Mp(£), absurd. 

Let us now consider the surjective homomorphism 7 : Sp(/c, £) + — > Sp(£ — 
k) given by the composition of w with projection onto the second factor. 
The following Lemma is left to the reader: 

Lemma 5. 

A 12 \ 


h-kj 

A e GL + (k), B e M M _ fc (M), A 12 e M k/ _ k (R), 

and An £ Mk(M), A22 € M^_fc(R) are both symmetric } . 

Thus, Z =: ker( 7 ) retracts to GL+(A;) C Sp + (k,£). Now s- 1 (GL(f)) C 
Mp(£) is simply the metalinear group ML(£), and consists of four connected 
components. Therefore, the inverse image s~ 1 (Z) C Mp + (/c, £) is the union 
of two connected components, each mapping isomorphically onto Z. We shall 
also denote by Z the identity component of s^ 1 (Z). 

Lemma 6. Mp + (A;, £)/Z = Mp(£ - k). 

Proof. The commutative diagram 

Mp + (M) 
I 



Mp + (M)/^ 
I 



Sp + (M) — ► Sp(£- fc) = Sp + (k,£)/Z 
exhibits Mp + (k, £) / Z as a connected double cover of Sp(£ — k). 

GL(fc) and Sp(£ — fc) sit naturally in Sp(k,£), namely as the commuting 
Lie subgroups of all matrices of the form 



(A 





and 



(h 






R 

T 











h 












s 





\ 




h-kj 



(A e GL(fc)) 



1? 
T 



5 
17 



G Sp(£ - fc) 
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respectively. We have therefore an injective morphism of Lie groups 

o = (C, x) ■ GL(fc) x Sp(£ - fc) - Sp(fc, £) C Sp(£). 

The inverse images s _1 (GL(fc)) and s _1 (Sp(£ — fc)) are isomorphic, respec- 
tively, to ML(fc) and Mp(£ — fc). Thus, there is a commutative diagram of 
Lie group homomorphisms 

ML(fc) Mp(fc,£) X Mp(^-fc) 

■J, J, J, 

GL(fc) Sp(fc,£) ^- Sp(^-fc). 

Taking products, we have a smooth map 

5 = (|, x) : ML(fc) x Mp(£ - fc) — >■ Mp(fc, t) C Mp(£), (<?, fe) ^ ffo) • ^(/i). 

Lemma 7. d is a Lie group homomorphism. 

Proof. Given that £ and x are morphisms, it suffices to show that 

1(9) ■ X(h) ■ i(g)- 1 ■ XW 1 = e, V (<?, fc) e ML(fc) x Mp(£ - fc). 

Given that o is a morphism, these commutators all lie in ker(s), which is 
finite of order 2. Given that each of £ and x is a morphism, they all equal e 
if at least one of g and fc is the identity. By connectedness of Mp(£ — k), the 
claim follows. 

The composition 

w o s| Mp(M) o 5 : ML(fc) x Mp(£ - fc) — ► GL(fc) x Sp(£ - fc) 

is clearly the product of the double covers ML(fc) — > SL(fc) and Mp(£ — fc) — > 
Sp(£ — fc), and is thus a 4:1 covering. 

Being contractible, T lifts isomorphically to a subgroup of Mp(£), that 
we still denote T < Mp(fc, £). Let us define the quotient group Mp(fc|£) = 
Mp(fc,£)/T. Let 9 : Mp(fc,£) -> Mp(fc|£) be the projection. On the upshot, 
we have: 
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Lemma 8. There is a commutative diagram of morphisms of Lie groups 



Mp(fc : £) — ► ML(k) x Mp(£ - k) 

5 

Mp(M) — ► Mp(fc|^) ( 9 ) 

si i 
w 

Sp(M) — > GL(k) x Sp(£ - k) 



where the upper square is a fibre product diagram, fi and v are 2:1 coverings, 
and uofM is the product of the 2:1 coverings ML(fc) — > GL(fc) and Mp(£— fc) — > 
Sp(£- A-). 

Recall that for every integer / > 1 the metalinear group ML(/) has four 
connected component. If r : ML(Z) — > GL(Z) is the projection, the compo- 
sition detor : ML(Z) — > R admits a square root, that we shall denote by 
V det : ML(Z) — > C. We shall denote the four connected components of ML(Z) 
by ML(Z)i, ML(Z)_i, ML(Z)i, ML(Z)_;, meaning that Vdet > on ML(Z)i, 
that Vdet is positive imaginary on ML(Z)j, and so forth. Clearly, ML(Z)i is 
the identity component, and r _1 = {e,g{\ where e is the identity and 
gi G ML(Z)_i. Given the standard inclusion ML(Z) Mp(Z), we shall view 
gi as sitting in Mp(Z). 

Lemma 9. ker(/i) = ker(o) = {e, (gk,9e-k)}, where e G ML(fc) x Mp(£ — fc) 
is the unit. 

Proof. Since fi is a 2:1 cover, it suffices to show that (gk,ge-k) £ ker(o). 
Since s o o((t/fc, ge-k)) G T = ker(ro), and T has no elements of order 2, 
s ° d{(9k, 9e-k)) is the identity. Given that o((gk, ge-k)) lies in ML(£), it 
suffices to notice that Vdet o o((gk, ge-k)) = (—1) • (—1) = 1- 

Corollary 2. By restriction of 5, we have an injective morphism of Lie 
groups GL + (k) x Mp(£ - k) ^ Mp(k,£) C Mp(£). 



Let us identify the unitary group U(£) with the maximal compact sub- 
group of Sp(£) consisting of all 2£ x 2£ symplectic matrices of the form 



X -Y 
Y X 



such that X + iY G U(f). 
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Notice that T n XJ(£) = {he} and Z n U(£) = O(Jfe). 

Similarly, let us set U(M) = Sp(fc, n XJ{£) and U+(M) = Sp + (M) n 
U(£), so that 



u(M) 



and 



/A \ 
i? 5 
A 
\ -5 i? y 

0(fc) x U(£-k), 



A e 0(k), 



R 

S 



-s 

R 



e U(£-k) 



U+(M) = SO(fc) x U(£ - k) 

is the identity component of XJ(k,£). 

Let MU(£) = s _1 (U(£)) C Mp(£). This is a connected compact sub- 
group of Mp(£). Similarly, let MU(M) = s _1 (U(M)) C Mp(fc,£). If we let 
MO(A;) = s^ 1 (0(£)), the vertical maps in the commutative diagram © both 
reduce to the composition of double covers 



MOO) x MU(£ - k) 
and the analogue of Corollary is 



MU(M) — ► U(M), 



Corollary 3. 5t/ restriction of o, we have an injective morphism of Lie 
groups SO(fc) x MU(^ - k) ^ MXJ + (k,£) C MU(£). 

For any / > 1, let «S(R ) be the space of smooth complex valued rapidly 
decaying functions on Mf, endowed with the bilinear pairing 



(/, g) 



f{x)g{x)dx (f,geS(R 1 )), 



and the L 2 -Hermitian product (f,g)h ='■ (fid)- The metaplectic group acts 
unitarily on (iS(M z ), (, under the Segal- Shale- Weyl representation, 

i; SS w : Mp(Z) — U (S(R 1 )) , 

which plays a crucial role in the symbolic calculus of Fourier-Hermite distri- 
butions. 

Given the inclusion of Lie groups of Corollary 121 (or Corollary EJ) , we may 
restricts the Segal-Shale-Weyl representation of Mp(£) to GL + (A;) xMp(i-k). 
On the other hand, GL+(A;) x MXJ(£-k) acts on <S(M^ fc ) = C® c S(R £ - k ) by 
the tensor product V det g) t^ssw of the charachter \/det on GL + (k) C ML(fc) 
and the Segal-Shale-Weyl representation of Mp(£ — k). Using the description 
of the metaplectic representation in |GS1| and |BG| . one can check that 
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Lemma 10. The restriction map S(R e ) — ► <S(1R k ), 

f(x u . . . ,x t ) h-> f TeB (x k+1 , ...,xi)=: /(0, . . . , 0, x k+1 , ...,x £ ) (f e S(R e )) 

is equivariant, that is, it is a morphism of GL + (k) x MU(£ — k)-modules. 

In the following, we shall use the concepts of metaplectic manifold and 
metaplectic vector bundle, a manifold being metaplectic if its tangent bundle 
is. If (E,Qe) is a symplectic vector bundle of rank 11 over a manifold N, 
we shall denote by Bp(E) — > N the principal Sp(f)-bundle of all symplectic 
frames in E. If the symplectic vector bundle (E, Qe) is metaplectic, we shall 
denote by Bp(E) the corresponding principal Mp(£)-bundle. 

In particular, if E admits a Lagrangian subbundle L C E, then E is 
symplectically equivalent to the vector bundle L © L*, with its standard 
symplectic structure. The structure group of E then reduces to GL(£) C 
Sp(£), and E is metaplectic if and only if L is metalinear. In particular, 
a cotangent bundle T*M is a metaplectic manifold if and only the base 
manifold M is metalinear, and furthermore any Lagrangian submanifold of 
a metaplectic manifold is metalinear. 

Let us now assume, more generally, that S C E is a rank-/c isotropic vector 
subbundle. The subbundle Bp(S', E) C Bp(E7) consisting of all symplectic 
basis whose first k vectors lie in S is a principal Sp(fc, £)-bundle over iV. 
Let BL(S') denote the principal GL(fc)-bundle over consisting of all linear 
frames in S. Let S ±n C be the symplectic annihilator of 5". We have 
BL(5) x Bp(5'- Lf2 /5') = Bp(5', E)/T, the projection being the bundle map 

(d, ...,e e ,fi,... fe) ^ ((e ls . . . , e £ ), ([e fc+1 ], . . . , [e/], [A+i], . . . , [ft])), (10) 

obviously equivariant with respect to the morphism of Lie groups (jSJ. 

Now suppose, in addition, that 5 is orientable. Let BL + (5') C BL(5) be 
the principal GL + (/c)-bundle of all oriented frames in S. Let Bp + (5 r , E) C 
Bp(S', E) be the subbundle consisting of all symplectic basis whose first k 
vectors form an oriented basis of S. Then Bp + (S*, E) is a principal Sp(/c, 
bundle over iV, and the projection Bp + (S,E) — > Bh + (S) x Bp(S'- Lf2 /S') is 
equivariant with respect to the morphism of Lie groups zu + in (JJJ). 

Proposition 1. Let (E,Qe) be a symplectic vector bundle, and let I C E 
be an oriented rank-k isotropic vector subbundle. Let I ±n C E denote the 
symplectic annihilator of S in E. Then there is a natural bisection between 
the set of equivalence classes of metaplectic structures on the symplectic vec- 
tor bundle Nj =: I n /I and the set of of equivalence classes of metaplectic 
structures on E. 
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Proof. In one direction, suppose given a metaplectic structure on E, that 
we describe by the following equivariant commutative diagram of principal 
bundles and double covers: 

Mp(f) x Mp(E) 
i 

Sp(£) x Bp(E) 
By restriction, we obtain the other 

Mp + (k,£) x Mp + (I,£) 

I 

Sp+(M) x Bp + (/,£) 

Given the projection 

Bp + (I,E) — > Bp + (/,E)/Z = Bp(iV / ), 

Lemmaininow shows that Mp(iVj) =: Mp + (J, -E)/^ is a metaplectic structure 
on JV>. 

In the opposite direction, suppose given a metaplectic structure on Nj. 
Let J{E, Qe) De the contractible space of all complex structures on E com- 
patible with fl E |MSj . and let us fix J G 0^). Thus E inherits a unitary 
structure, say h = g + iQ where g(v,w) = fl(v, Jw) and, by restriction of 
g, S is a Euclidean vector bundle. Let I ±9 and I ±h be the Euclidean and 
Hermitian orthocomplements of / in E. The intersection I ±h = I ±9 D J -1 " 
is a complex vector subbundle of E, linearly symplectomorphic to the vector 
bundle iVj. Having this identification in mind, let BU(iVj) be the bundle of all 
unitary frames in Nj. The metaplectic structure then yields an equivariant 
commutative diagram 

MU(Jfe) x BU(iVj) — ► BU(iVj) 

I I 

U(Jfe) x BU(iVj) — ► BU(JV». 

Let BO_|_(7) C BL_|_(7) be the subbundle of all oriented orthonormal frames 
of I. Let EU+(I,E) = BX1(E) n BL + (7, E). We have an isomorphism of 
principal U+(fc, £)-bundles 

Bp(£) D BU+(J,£) = BO+(J) XjvBU(iVj). 



Mp(£) 
I 

Bp(^). 



Mp + (J,£) 
I 

Bp + (/,£). 
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Now BU + (J, E) = BO_|_(7) x N BXJ(Nj) is an equivariant double cover of 
BU + (I,E) with respect to the double cover of structure groups 

SO(A;) x MV(£ - k) -> U+(Jfe, £), 

and we obtain a metaplectic structure of E by group extension in view of 
Corollary El 

It is clear that these two correspondences are one the inverse of the other. 

Corollary 4. Let E be a metaplectic vector bundle of rank 21. Let S C E 
be an oriented rank-k isotropic subbundle. Then the structure group of E 
reduces to GL + (A;) x Mp(£ - k) C Mp(f). 

If E is metaplectic vector bundle of rank 2£, given the Segal-Shale- Weyl 
representation we may form the associated infinite dimensional vector bundle 

S(E) =: Bp(E)x Mp{e) S(R £ ). 

If / C E is an oriented rank-A; isotropic subbundle, in view of the induced 
metaplectic structure on Nj we may similarly form infinite dimensional vector 
bundle 

S(Nj) =: Bp(JV» x Mp( ,_ fe) S(R £ - k ). 

We have: 

Corollary 5. Let E be a metaplectic vector bundle of rank 2£, and let I C 
E be an oriented rank-k isotropic subbundle. Then the restriction map of 
Lemma extends to a surjective morphism of vector bundles 

-1/2 

$i : S(E) — > /\(I)® c S(Ni). 

By the bilinear pairing ( , ), we may view in the standard manner 5(]R £ ) 
as a subspace of the space ^'(IR^) of tempered distribution, which is invariant 
under the dual representation of Mp(£) on <S'(M ). If E is as above a rank- 
2£ metaplectic vector bundle, let Sd ua ,i(E) denote the vector associated to 
the metaplectic structure and the dual action of Mp(£) on iS(IR £ ). Since the 
metaplectic representation is unitary, we have an isomorphism 

Sdual{E) = S(Ej. 

Passing to conjugate bundles and inserting this isomorphism, we obtain from 
Corollary El a surjective morphism of vector bundles [BGJ 

Sdual(£) — > A 1/2 (J)®C.Wtfj). (11) 
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Consider now the inclusion of Lie groups 

SO{£) C GL + {£) C ML(£) C Mp(£). 

The function e~^ 2 = e - ^*? G «S(R € ) is a fixed point for the restriction 
of the Segal-Shale- Weyl representation to the subgroup SO(£). In the spe- 
cial case where E admits an oriented Lagrangian subbundle L, as we have 
mentioned the structure group reduces to GL + (£). The choice of a com- 
patible complex structure J G J(E,VL E ) further reduces it to SO(£), the 
corresponding principal SO(£)-bundle Pl,j being the bundle of all frames of 
the type 

(d, . . . ,e t , Jp(ei), . . . , J p {ei)), 
where (ei, . . . , eg) is an oriented orthonormal frame for L(p), p e N. Thus: 

Lemma 11. Suppose that the symplectic bundle (E,Qe) has an oriented 
Lagrangian subbundle L C E. Then to any J G J{E, Qe) there is associated 
a natural nowhere vanishing section o~l,j of S(E), which may be described 
as the constant S(Mf) -valued function on Bl,j equal to e - " 1 " . Its image in 
fi _1 / 2 (L) corresponds to the constant function 1 on B LJ . 

In a related vein, the theory of |BGj also shows that the function e~" x " 2 
is a joint eigenvector for the metaplectic action of MU(£) =: s _1 (£/(£)) C 
Mp(£), i.e. there is a unitary charachter c : MU(£) — > U(l) such that 

v ssw (A) (e-" x H 2 ) = c(A) ■ e-" x " 2 (A G MU(£).) 

Thus, the function e - " x " 2 ~" y " 2 is a fixed point for the tensor product action 
of MU(£) on S(R l ) ®S dual (M. e ) = S(R l ) ®S(W). It represents the orthogonal 

projection of <S(E^) onto the subspace span je~" x " 2 j. 

Now the choice of a compatible complex structure J G J7(E, Qe) reduces 
the structure group of E to U(£); let Bu(E, J) C B\(E) be the principal 
U(£)-bundle of all unitary frames in E (for J)). If (E,Q E ) in addition 
is metaplectic, with metaplectic strucure Bp(E) — > Bp(E), the inverse image 
Bu(E, J) =: zu^ 1 (Bu(E, J)) is a principal MU(£)-bundle. Thus, we may 
globalize the previous construction to obtain: 

Lemma 12. Let (E,Qe) be a metaplectic vector bundle. Then to each J G 
J~(E,Qe) there is associated a line subbundle Lj C S(E), which in the 
trivialization of S(E) offered by any element ofBu(E, J) is the line spanned 
by e _ " x " . In the same trivialization, the orthogonal projector S(E) — > Lj, 
as a section of the bundle of linear endomorphisms ofS(E), is represented 
by the function e~" x " ~" y " . 
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When operating on symbols of Szego kernels and related distributions, 
a naturally occurring case is that of symplectic vector bundles of the form 
E + © E~ , as in the following Lemma (that we simply state): 

Lemma 13. i) Let E + = (E,Q E ) o,nd F + = (F,Q E ) be metaplectic vector 
bundles on a manifold N. Then there is a naturally induced metaplectic 
structure on their direct sum E + (BF + = (E®F, Q e (BQf), and a natural 
isomorphism 

S(E + @F + ) = S(E)®S(F). 

ii) Let E + = (E,Q E ) be a metaplectic vector bundle. Then there is a na- 

turally induced metaplectic structure on its opposite E~ =: (E,—fl E ), 
and a natural isomorphism 

S dual (£+). 

iii) In particular, if E + = (E, Q E ) is a metaplectic vector bundle then there 

is a naturally induced metaplectic structure on E + ®E~, and a natural 
isomorphism 

S(E+ © ET) ^End ns (S(E)), 

where the latter denotes the vector bundle of Hilbert- Schmidt linear 
endomorphisms of S(E). 

US = E + ®E-, for any J G J{E, Q E ) we have 

j s =:j® (- j) g j (s, n E © (-n E )) . 

Any unitary frame (ei, . . . , eg) G Bu(.B, J) = Bu(£', — J) extends to a uni- 
tary frame (ei, . . . , eg, e\ . . . , ei) G Bu(iV, J^)- Passing to metaplectic double 

covers, Bu(E, J) C Bu(5', Js). Summing up, we have: 

Corollary 6. Suppose that the symplectic vector bundle (E, Q E ) is meta- 
plectic. Set S =: E + © E~ . Then S(S) = End-^ l s(5(i?)) has a distinguished 
section o~j for every compatible complex structure J G J(E, fl E ). In any ap- 
propriate trivialization (in the sense above), this is represented by the function 
e -\\xf-\\Y\\ 2 eS {R 2i ). 

We now look at the image of the distinguished section in CorollarylHlunder 
the morphism of vector bundles in Corollary |S| The proof of the following is 
a case by case application of the local form in Lemma ITU1 of the vector bundle 
morphism of Corollary and is left to the reader: 
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Corollary 7. In the hypothesis of Corollary® suppose that L C E is an 
oriented isotropic subbundle, and consider the isotropic subbundle diag(L) = 
{(I, I) : I G L} C S . Fix J G J(E, Qe), and let h be the Hermitian structure 
on E associated to the compatible pair (Qe,J)- Let g = ^t(h) be the asso- 
ciated Riemannian metric. Let L 1 - C E be the symplectic annihilator of L 
in (E,Qe), and let L° be the Riemannian orthocomplement of L in (E,g). 
Then: 

i) : The intersection L^-CiL is a complex vector subbundle of(E, Qe); whence 

a symplectic subbundle of (E,Qe) / 

ii) : The symplectic normal bundle Nl = L L /L of L in E + is (naturally) sym- 

plectically isomorphic to L L R L° (we shall henceforth not distinguish 
between Nl and L 1 - R LP ), and thus has a naturally induced compatible 
complex structure Jn l ; 

iii) : Let us endow S = E + © E~ with the compatible complex structure J © 

(— J). Then the symplectic normal bundle iV" diag ( L ) of diag(L) in S is 
(naturally isomorphic to) the direct sum of vector subbundles 

iVdia g (L) = ({{l,-l):lEL}®{(Jl,Jl):leL}) 
©(iV+ffiiV L -) 
= (L r ®L i )®(N+®N L ) 1 > 

= L c © (iV+ © N£) . 

Here L r and L i} defined by the second identity, are oriented Lagrangian 
subbundles of the complex vector subbundle Lq =: L r © Lj C S. Since 
L r = L is oriented, Lq is metaplectic. 

iv) : Let $dia g (L) : S(S) — > /\ -1 ' 2 (L) ®S (N^^l)) be the vector bundle mor- 

phism introduced in Corollary® Let oj be the distinguished section of 
S(S) associated to the complex structure J as in Corollary® Then 

$dia g (L) (oj) = vol L © a Lr © a,j NL . 

Here vol^ 1 ^ 2 is the section of /\^ 1 ^ 2 (I) taking value one on oriented 
orthonormal basis of I, ol t is the section of <S(Lc) associated to the 
Lagragian subbundle L r according to Lemma [TJ[ and o~j Nl is the sec- 
tion of S(N£ © N£) associated to the complex structure Jn l on Nl, 
according to Corollary® 

v) : The symplectic normal bundle of L r in A r di ag (L) is isomorphic as a unitary 

vector bundle to N+@N L . Let $ £r : S(N diag(L) ) -> /\~ 1/2 (L r )®S (N Lr ) 
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be the vector bundle morphism from Corollary^ Then 
$L r (o- Lr ® o- jN ^j = vol~ r 1/2 <g> a jNL . 

3 The geometry of the symbol calculus 

We need to recall some basic constructions from |BGj . at places rephrasing 
them in terms of the principal bundles involved in our constructions. Let 
A and B be C°° manifolds. There is a natural symplectomorphism T*(A x 
B) = T*(A) x T*(B) (cotangent bundles are implicitly endowed with their 
canonical symplectic structures), which will be implicit throughout. Suppose 
that T' C T*(A x B) \ {0} is a closed Lagrangian conic submanifold. Let 
T C T*(A) x T*(B){0} be the associated canonical relation, defined as the 
image of T' under the involution ((a, r/), (6, ■#)) i— > ((a,r/), (6, — i?)J. Suppose 
that E C T*(B) \ {0} is a closed isotropic conic submanifold. Let us form 
the fibre diagram 

P 



f — ► r 

Q i i 1 
i 

E — > T*(B)\{0}. 



Here i is the inclusion, and 7 the projection. Thus, 

F = {(a,(r,a)) : a G E, (r, a) G T} . 

We have a diffeomorphism 

F = gf^E) =g- 1 (E)nr, (a, (r, a)) <- (r, a), 

so that F maps naturally into T, diffeomorphically onto its image. We shall 
implicitly think of F as embedded in T, as the subset of all pairs (r, a) G T 
with a G E. 

Let us assume that all the clean intersection and properness hypothesis 
in Chapter 7 of loc. cit. are satisfied (this will be always the case in our 
situation). Given the projections 

T T*(A x B) 

p F qr P q 

/ \ / \ 

T*(A) T*(B) T*(A) T*(B) 
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we have a conic isotropic submanifold 

ToS = {t G T*(A) \ {0} : 3a G E such that (r, a) G T} 

= P (g- 1 (E)nr) 

= p r (g r 1 (s))cr(A)\{o}. 

Furthermore, the projection p F : F -» T o S, (a, (r, a)) h- >■ r, is a fibration 
with compact fibres. 

Let us define a vector bundle Uq on F by 

E7o(r, a) = {we T CT E : (0, iy) G T (T>ff) r} . 

Since is a submersion, if G t/o( r , o") then (0,w) G Tt T ^F is tangent to 
the fibre of pp over r. In other words, TF D Uo = ker(dpp) is the vertical 
tangent bundle of pp. 

The functorial behaviour of the symbol of Fourier-Hermite generalized 
half-forms is governed by a symbol map, which transforms symplectic spinors 
on E into symplectic spinors on r o E, and whose existence is the content 
of Proposition 6.5 of |BG| . More precisely, in the present context this is a 
surjective morphism of vector bundles on F, 

^s,r : P* |A( r ) j ® c Q * ( S P in ( S )) — > det (^o) ® P*F (Spm(r o E)) . 

In the sequel, we shall need to reformulate its construction, in order to have 
an explicit description of the symbol map in terms of the principal bundles 
involved. 

Since in our applications A, B, T, E, F, the vector bundle U above and 
the vector bundle U introduced below will all be orientable, we shall make 
this simplifying assumption throughout. It will also simplify our exposition 
to assume, as will be the case in our applications, that pp : F — > V o E is 
a Riemannian submersion, having therefore a natural connection, and that 
TF has a natural (oriented) complement in TF\ F , denoted Np\r- 

Given that B is orientable, hence metalinear, T*B is metaplectic. Since 
E C T*B \ {0} is an oriented isotropic submanifold, its symplectic normal 
bundle in T*B, = (TE)- L /TE, has an induced metaplectic structure. The 
spinor bundle of E is 

1/2 

Spin(E) = /\(E)®S(iV E ). 
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The same considerations apply to the conic isotropic submanifold T o E C 
T* A \ {0}, with associated spinor bundle 

1/2 

Spin(r o E) = /\(r o E) <g> S(iV roS ). 
Let us define a second vector bundle U\ D U$ on F by 

U x {r,a) = {we (TaE) 1 - : (0, w) G T {T:a) T} , 

for (r, a) G F. 

Then Uq and C/j are isotropic vector subbundles of the symplectic vector 
bundle g* (T(T*B)), and their quotient U = U\/Uq C q* (AT s ) is an isotropic 
vector subbundle, which as mentioned we shall assume oriented. By Corollary 
El therefore, the structure group of the metaplectic bundle of iVx, Bp(iVs), 
reduces to 

Bl+(17) x BpiU^/U), 

and there is furthermore a surjective morphism of vector bundles associated 
to this reduced principal bundle, 

-1/2 

G^u : <S(iV s ) — /\ (U) (8) S {U^/U) . 

By construction, Gj] t u is in the appropriate trivialization the restriction map 
of Lemma El 

By Proposition 6.4 of |BG| . iV roS = U^/U naturally, so that there 
is at any rate a morphism of vector bundles on F from p* ^A^ 2 (r)j ®c 
q* (Spin(E)) to 



P 



1/2 \ /l/2 \ -1/2 

A(n ® e*[ A( s ) }® /\(u)®p* F (<s(iv roS )) . (is) 



We shall therefore be done by exhibiting an isomorphism of line bundles 



P 



1/2 \ £ 1/2 \ -1/2 /l/2 



For the rest of this proof, we shall fix a point / = (r, a) G F, and write 
for notational simplicity TE, TF, and so forth for TfF, TfT, and so forth. 
Similarly, U , Ui, U will be the fibres at / of the corresponding vector bundles 
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on F, and the same provision will apply to the various principal bundles 
involved. 

Given the assumptions discussed above, we have direct sum decomposi- 
tions 

TV = TF © iV = U © T(T o E) © N, 

whence we may reduce the principal bundle of the line bundles on both sides 
of (fTlj) to the product 

Bl+(U ) x Bl+(T(r o £)) x B1+(JV) x B1+(E) x Bl+(U), 

whose general element we shall denote by (eu , er s> ejv, e^, ey). 

Let Ui be the symplectic annihilator of U\ in T(T*B). Let 7 : 7T©T£ — > 
be given by j((v, w),w') — w — w'. We then have a short exact sequence 

(HD], page 45): 

— > TF — > TT © TE — ^ Ui — > 0. 

Under 7, the subspace N © TE C TT © TE maps isomorphically onto U(- = 
(T(T*B)/U 1 )*. Given a pair ( ejv , e E ) G Bl+(iV) x Bl+(£), we shall denote by 
f(e N , es ) the oriented basis oiT(T*B)/U\ dual to the oriented basis 7 ((ejv, e^)) 
of (T(T*B)/Ui)*. 

Let fiT*s,can denote the canonical symplectic structure of T*B, vo\t*b = 
^T*B,can the associated volume form, and voly* 2 B = yj Qt*b, can the associated 

half-form. If now we are given tf r G A 1/2 ( r )> G A V2 ( S )> ^ G /\~ 1/2 (U), 
the expression 

1 1/2 

$r(er/ , e ro s, e^) • $s(e s ) • &u ( e u) ' volj. B {e Uo} e v , f(e N ,e s )) (15) 

only depends on (eu , eros) G Bl + (f/ ) x B1 + (T o E). 
We thus have: 

Proposition 2. Assume that Uq is oriented. There the previous construction 
defines a surjective morphism of vector bundles on F 

#E,r : P* |y\( r ) j ® c ( S P in ( S )) — > det TO ® Pf (Spm(r o £)) . 

Our additional hypothesis that Uq be orientable accounts for the appear- 
ance of the determinant line bundle in place of the line bundle of densities. 



21 



4 Restricting and pushing forward Tlx- 

Let L* be the dual line bundle to L, endowed with the induced Hermitian 
metric. Let X C L* be the unit circle bundle, H(X) C C°°(X) be the Hardy 
space of boundary values of holomorphic functions. The S^-action induces 
a decomposition H(X) = @ fceN Hk(X), and the the k-th isotype iffc(X) 
is canonically isomorphic to H°(M, L® k ). Given that Vt is symplectic, the 
connection form a is a contact structure on X. Given that Hermitian metric 
h on L is G-invariant, the action of G on L* leaves X invariant. 

The Kahler manifold (M, Q) and the contact manifold (X, a) have natural 
volume forms voIm = ^ An and volx = ^x( vo ^m) A a, respectively. Let 
voljy 2 ' 1 = VvoIm and vol x ^ 2 ^ = ^/volx denote the respective associated half- 
forms. The volume form volx makes the space of smooth functions on X 
into a prehilbert vector space, unitarily isomorphic to the space Q}I 2 (X) of 
smooth half- forms under the map / h- > f vo\^ 2 \ This isomorphism will 
be implicit throughout, and we shall accordingly view the Szego kernel tlx, 
rather than as a generalized density, as a generalized half- form on X x X: 
Tlx G Vy 2 (X x X). Similarly, taking products we obtain volume forms 
voImxm and volxxx on M x M and X x X, respectively, with associated 

half- forms voI^m an d v0 ^xxx- 

In general, if fi and f% are functions, or sections, or half-forms, and so 
forth, on manifolds A\ and A2, we shall denote by j\ Kl fi = 7r*(/i) ® Tttifz) 
the corresponding object on the product Ai x A 2 obtained by pull-back under 
the projections 7i~j : A 1 x A 2 — > A, and tensor product. Thus, we may write 
the Szego kernel as 



where for every k = 1,2, . . . |s^- j (1 < j 1 < h°(M, L® h )) is an orthonormal 

basis for the k-th Fourier component H(X)k — H°(M,L® h ) of the Hardy 
space H(X). 

Let W =: ^-^O) CM. If tt x \m ■ X -> M is the projection, let X' =: 
7r^ M (M') and denote by jx> : X' ^ X be the inclusion. Under the above 
assumptions, px> '■ X' — > Xo =: X'/G is a principal G-bundle. Clearly, Xo is 
the unit circle bundle in Lq with the induced metric ho. We shall now endow 
the inclusion j x > and the projection p X ' with the structure of morphisms of 
metalinear manifolds; by taking products, this will also make the inclusion 
jx'xx' '■ X' x X' — > X x X and the projection 7Tx' : X' x X' — > X x X into 
morphisms of metalinear manifolds. We shall then apply the associated pull- 
back and push-forward operations to the Szego kernel Ilx G T>' 1/2 (X x X). 



fix = J2 s ? ) ® s ? ) -™ i x 2) ®vol x /2 \ 
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By using the microlocal theory of [BG , we shall relate the result to the Szego 
kernel of the symplectic quotient X , U Xo £ T>' x , 2 (Xq x X ). 

To this end, let us fix an orientation on G, and thus on its Lie algebra, 
q. In particular, this makes M' and X' into oriented Riemannian manifolds. 
As discussed in section 2, there is then a natural way to restrict a half- 
form on M (or X) to a half- form on M' (respectively, on X'), namely by 
setting l*{t]m) — Vm' (respectively, f(r] X ) = f]x')- This makes i and j into 
morphisms of metalinear manifolds, Z and j. 

Taking products, we clearly obtain restriction maps i* ® i* and f ® j* on 
half-form bundles and (t x l)* : Vt l l 2 (M x M) -> fiV2(jvf' x M ^ ^ x 3 y . 
fl x /2(x x X) -f fi^X' x X') on smooth global half-forms. 

Let T JXJ C T*(X x X) x T*(X' x X') be the canonical relation associated 
to j x j. The morphism of metalinear manifolds J x J is equivalent to the 
assignment of the half- form vol]/ 2 described in (jHJ), with / = j x j, P = 
X' x X' and Q = X x X. 

By the same token, the principal G-bundle px> '■ X' — > X = X'/ G is also 
a morphism of metalinear manifolds in the natural manner: p X '{Vx ) = Vx>- 
Let px>* '■ fi x ' 2 (X') — > fi 1 / 2 (X ) denote the resulting push-forward operator. 
For any smooth function / on X', let us denote by f G the G-invariant compo- 
nent of /, when decomposed over the irreducible representations of G. Thus 
f G may be viewed implicitly as a function on Xo = X'/G. Furthermore, let 
us write V e s for the effective potential of the action, that is, V e s(x) is the 
volume of the fibre p x )(x), x £ X [BG]. By Lemma [TJ if x £ X then 



Taking products, the principal G x G-bundle px'xX' = Px< x Px' : 
I'xI'-> Xo x Xo can be similarly made into a morphism of metalinear ma- 
nifolds, with an attached push- forward operation px'xx 1 

: &/\X' x X') -> 

f2 1//2 (XxX). If/, g are smooth functions on X' and (fMg) (x, y) =: f(x)-g(y) 
(x, y £ X'), then (/ M g) GxG = f G Kl g G . Furthermore, the effective poten- 
tial for the product is = V cS K V eS , that is, V^(z,y) = V cS (x) ■ V eS (y) 
(x, y £ X ). 
Thus, 

Lemma 14. vol^x' = v °1y"' ^ Y °^x' 2 \ an d 





f G (x) • Kff (x) ■ vol 
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for any pair of smooth functions f and g on X . 
Proof. This follows from the equalities 

Px>xX>* (/ E^vol^,) = px>xx>* ((/ • vol^ 2) ) El (g vol^ 2) ) 

= (f G V cS vo\^)M(g G V cS .vo\^] 



Since px'xX' is proper, the push-forward px'xx 1 * extends to a continuous 
linear map of Frechet vector spaces 

Px'xX'* : V' 1/2 (X' x X') — X^ /2 (X x X ). 

Let furthermore £ C T*(X x X) \ {0} be the wave front set of the Szego 
kernel, that is, 

S = {(x, x, ra x , —rax) '■ x G X, r > 0} . 

Let 2X/ 2 (X x X)s C T>' X , 2 {X x X) be the subspace of all generalized half- 
forms on X x X having wave front contained in E, with the appropriate 
topology. Let 

N j*j = {(^1,^2,7/1,^2) : Xi G X', r/j = on T X X' C T Xi X} 

be the conormal bundle of the embedding j x j. Since S n N JXJ = 0, the 
pull-back (j x j)* extends to a continuous linear map of Frechet vector spaces 
(|Dj. Proposition 1.3.3) 

(j x j)* : V[ /2 (X x X) s — Pi /2 (X' x X'). 

By composition, we thus have a well-defined continuous linear map 

T =: Px'xx'* o {j x j)* : V' 1/2 (X x X) s D 1/2 (X x X ). 

Let us define, for fceZ, 

V' 1/2 (X xX ) k = {(p G T>'(Xq x Xo) : 

u(e i6 x,y) = e lke u(x,y)\/e ie eS 1 ,Vx,yeX}, 

and similarly on P' 1/2 (X x X) and V[JX' x X'). Let T k : P' 1/2 (X x X ) — ► 
X^ /2 (X x X ) fe be given by 

-2tt 



T k {u) = i- J\- ik9 u{e i6 x iy )d6. 
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Then 

+00 

V' l/2 (X x X ) = V' l/2 (X x X ) k , 

k=—oo 

and Tk represents the projection onto the /c-th factor. 
We now focus on the generalized half-form 

tl x \x =: T (fl x ) G V' 1/2 (X x X Q ). 

Since j and px> are S^-equivariant maps, we have for every fceZ: 

T fc (r (rV)) = r (r fc (n x )). 

Clearly, T^ILx) is the distributional kernel of the orthogonal projector onto 
the subspace H(X) k = H°(M,L® k ). 

For every k = 0, 1, 2, • • • let <! s, G i denote an orthonormal basis of 

the space of G-invariant Hardy functions of level k, H(X)^ = H°(M, L® k ) G . 
By Lemma ITU we have 

+00 d k 

fixivo (x ,2/o) = Kff(x )Kff(l/o) E53 s f ® s f' fc V)> ( 16 ) 

fc=o i=i 

if xo, yo ^ X an( i x ' 1 y' li e over ^Oj Uo respectively. 

We shall now use the theory of [BGj to describe Hx\x as a Fourier- 
Hermite generalized half-form. Namely, let us first recall that, as a Fourier- 
Hermite generalized half-form, the Szego kernel satisfies Tlx € J l l 2 (X x 
X, X). Let furthermore «o be the connection 1-form on Xq, and set 

S = {(x , X, TO£q x , TO>q x ):xeX , r>0}CT*(X xX )\{0}. 

Lemma 15. fl x \x G J( 1+9 )/ 2 (X x X ,S ). 

Proof. Referring to chapter 7 of |BGj . let us consider the fibre diagram 
associated to X C T*(X x X) \ {0} and the conormal bundle F' JXJ C T*(X' x 
X' x X x X) \ {0} of the graph of j x j: 

I I (17) 

E — ► T*(X x X), 
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where r jxj denotes the image of 1^ under sign reversal in the first compo- 
nent of T*((X' x X') x (X x X)). We have 

r jxj = {(x 1 ,X2 J 3(x 1 ),j(x2),(d Xl j) t (r] 1 ),(d X2 j) t (r] 2 ),r] 1 ,r] 2 ) : 

x i eX',T H e7% ai) (X),i = i,2.}. (18) 

Let us set a' =: f(a). Then 

F jxj = {{ x ^ x ^i.x),]{x),ra' x , -ra' x ,ra j{x) , -ra ]{x) ) : x G X', r > 0.} . 
Thus F JXJ is diffeomorphic to its projection 

E' = T JXJ o E = {(x, x, ra' x , -roQ : x G X', r > 0.} C T*(X' x X') \ {0}. 

Therefore, the excess of the diagram ()17j) is 

e JXJ = dim(F 3Xj ) + dimT*(X x X) - dim (T' JXJ ) - dim(E) 
= 9- 

Hence, given that all the clean intersection hypothesis of Theorem 9.1 of [BGJ 
are satisfied, we have (j x j)*(fl x ) G J( 1+ ^ 2 (X' x X', E'). 

Let us next consider the fibre diagram associated to px> x Px'- The 
canonical relation is now 

r jxj = {(^l^px'O^iXPx'O^), (d Xl p X 'Y(r]i), (dx 2 Px>) t (r)2),VhV2) '■ 

x t ex>, r ]l eT; xr(Xi) (x ), i = i,2.y 

(19) 

The fibre diagram is 

I I (20) 

s' — > r(i'xi'). 

Here -F Px;XPx , is clearly diffeomorphic to E'. Thus, the excess is now 

e Px ,x Px , = dim(F Px , XPx/ ) + dimT*(X' x X') - dim (T Px , XPx ,) - dim(E') 
= 2g. 

Given that the fibres of px' x px> have dimension 2g, Theorem 9.2 of loc.cit. 
now implies (p x , x p x ,)*(j x j)* fix) e J( 1+ ^/ 2 (X x X , E ). 
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5 The symplectic structure along the cone. 



Let 

S a = {(x, ra x ) : x E X, r > 0} C T*(X) \ {0} (21) 

be the half-line bundle generated by the connection 1-form. This is a closed 
symplectic cone in T*(X). Let p : T, a — > X and g = 7rop:S Q! ^Mbe 
the projections. Since S a is naturally diffeomorphic to X x R+, we have an 
intrinsic isomorphism of vector bundles on E a , 

T(E Q ) = p*(TX)© span 1 1- 

where £ denotes the generator of the Reaction on S Q . On the other hand, 
the connection 1-form induces a splitting T(X) = n*(T(M)) © span{^}, 
where ^ denotes the generator of the ^-action. On the upshot, we have an 
isomorphism of vector bundles on S a : 

T(E a ) - q*(T(M)) © span (I- , |-1 . (22) 



<9r <96* 

Since X is oriented, it is metalinear and therefore T*(X) is metaplectic. Thus 
T(T*(X)) is a metaplectic vector bundle, and therefore so is its restriction 
T(T*(X))| Ea . We shall now examine the situation in local coordinates along 
S a . 

Fix z = (x,ra x ) E S Q and set p = n(x). Let U C X be a coordinate 
neighbourhood for X near x. Thus T*£7 C T*X is an open subset, and 
there is an obvious symplectomorphism T*U = U x R271+1, where the latter 
is viewed as an open subset of R 2n+1 x R 2 n+i ) with its natural symplectic 
structure (here R 2n +i = (R 2n+1 )*). We shall identify R 2n +i with R 2n+1 by 
means of the standard scalar product, and view a\ v as a smooth R 2n+1 -valued 
form on U . Let Jac 2/ (a) be its Jacobian matrix at y E U . 

A differential 2-form on U is represented by smooth function U — > A 2n+ i, 
where A2 n +i is the space of antisymmetric matrices of order 2n+ 1. Namely, 
if we denote by tj the local coordinates on U, the differential form v = 
Vijdti A dtj is represented by the A 2n+1 -valued function \y^j\. The proof 
of the following is left to the reader: 

Lemma 16. The matrix valued function \ [Jac(a)* — Jac(a)] represents 7r* (fi) 
on U. 



In other words, if y E U and v — ^ Vi -§f , w = J2i w i~§T e T y (X) then 



» 1/4 at. 



u 



7T*(Q) y (v,w) = \v l [Jacket)* — Jacket)] w. 
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Corollary 8. Let V = ^ Vi-J^- be a vector field on U. Then the contraction 
l{V)tt*{Q) = 7r*(fl)(V, •) is the 1-form represented on U by the vector valued 
function | [Jac y («) — Jac 2/ (a)*] V. 

Lemma 17. The pull-back q*{TM) has a natural metaplectic structure. 

By (I22j) . Lemma IT7I implies that S has a natural metaplectic structure. 
Proof. With some abuse of language, in the symplectic coordinate chart U x 
R 2n+1 the tangent space T z (E a ) is the symplectic subspace of M 2n+1 x M 2n+1 
given by 

T ^ - {UcImJ- 6 ^ 1 }®^!©} 

© span | ^ja^Q^J > | > 

where k represents Clearly, is just (j22]) in local coordinates. It follows 
that the symplectic orthocomplement of T z (H a ) in T Z (T*(X)) is given by 

T ^ ± ={(rJ I ( a )v) :VEb>lMQR2 '' + '}- <24) 

For simplicity, let us use g*(TM)h om as a short hand for the symplectic vec- 
tor bundle (q*(TM),2r q*(Q)) (r > is the conic coordinate on S a ). Then 
pij) shows that the differential dq induces a symplectic isomorphism between 
the vector subundle T(S Q ,)- L and the pull-back g*(T(M))h om (i-e., q* {T M\ oni 
with the opposite symplectic structure). On the other hand, rescaling pro- 
vides a symplectic isomorphism g*(TM) hom = q*(TM) (the latter as a short 
hand for (q*(TM), q*(Jl))). . On the upshot, we have a symplectic isomor- 
phism 

T(T*X)| Sa = g*(TM) hom ©g*(TM)- om ©span{|;,f } 

S g*(TM) ©g*(TM)" © span 1 ° J 

Let now Bp(g*(TM)) = g*(Bp(TM)) and Bp(g*(TM)") be the principal 
Sp(n)-bundles of all symplectic frames in q*(T*(M)) and q*(T*(M))~, re- 
spectively. Let the automorphism a : Sp(n) — > Sp(n) be defined by 

*W = : ( o -/„) " (o -I) iU e Sp(n)) ' 
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Let us redefine the action of Sp(n) on Bp (q*(TM) ) by composing with a. 
Then the map r : Bp(g*(TM)) -> Bp(g* (TM) - ) given by 

r : (ei, e„, / x , ...,/„) (-ei, . . . , -e n , f u . . . , f n ) 

is an Sp(n)-equivariant diffeomorphism. We then have an Sp(n)-equivariant 
embedding r : Bp(g*(TM)) -> Bp(T*X| s J given by 



(e,f) 



f f 9 9 
e i — e i — — 

' dO 1 dr 



for (e, f) = (e 1 ,...,e n ,f 1 ,...,f n ) G Bp(<f(TM)). The inverse image of 
t : Bp (q*(TM)) in the metaplectic cover of Bp(T*X| E ) is the asserted 
metaplectic structure of q*(TM). 

Let us now fix £ = ( Ob ^ Ob . fGi^^ ITQtrQ ) G S and examine the symplectic 
structure of T*(XxX) near (• We shall also identify T*(XxX) with T*(X) x 
T*(X). In the symplectic product coordinate chart (U x R 2n+1 ) x (U X 
R 2n+1% ) the tangent space T^(E) is the isotropic subspace of (R 2n+1 x R 2n+1 ) x 
given by 



o2n+l y "p2n+l x 



T C (E) 



/ « \ 

rJac x (a)v 
v 

\-rJ x (a)vJ 



: v G 



o2n+l 



span 



/ \ 

a x 


v-«*y 



The symplectic annihilator of T^(E) is then 



T C (E) 



/ « \ 

rJac x (a)*v 

^— rJac 1 .(a)*w y 



: atxiv) = a x (w) = > + T C (E 



(26) 



Lemma 18. The sum of vector spaces on the left hand side of 1126]) is direct. 



Proof. Let a, b, c G R, xf G R 2n+1 and v, w G ker(a x ) C R 2n+1 be such that 



/ 



\ 



rJ&c r (a)v' 



\— rJac x (a)v' J 



( v \ 

rJac x (aYv 
w 

\— rJac x (aYw J 



+ c 



( \ 



\-a x J 



0. 
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If a or b vanish then so do all the other coefficients. We may otherwise absorb 
them into v and w sot as to assume a = — b = 1; then v' = v = w G ker^). 
Thus we are reduced to the equality 

(Jac x (a) — Jac a .(a)*) v — — a x , 

for a certain v G ker(a x ). Now ker(ctj;) is the horizontal subspace for the 
connection, and the skew matrix ~ [Jac x (a)* — Jac rE (o;)] represents the 2-form 
d x a = n*(tt) x . 

Claim 5.1. Let fi be a 2-form on M and let IV be a vector field on M. 
Denote by W the horizontal lift of W to a vector field on X, under the 
given connection. Let i be the contraction operator between vector fields 
and differential forms. Then 

l(W)tT*(ji) = 7T*(i(W)/i). 

In fact, both 1- forms vanish on vertical tangent vectors, and they obviously 
take the same values on horizontal vectors. 

Thus, if v represents (in our coordinate patch) the horizontal lift of a 
tangent vector £ G T p (M), p = ir(x), then | [Jac x (a) < — Jac ;c (a)] v represents 
the pull-back of the 1-form t(£) Q p under the differential d p n : T X (X) — ► 
T p (M). But this may not be a multiple of the connection, unless it vanishes. 
Thus, c = and (Jac x (a) — Jac x (a)*) v = 0, that is, Q p = 0. This 
contradicts the nondegeneracy of Q, unless £ = and therefore v — 0. Lemma 
EU follows. 

Therefore, the vector subspace 



/ « \ 

rJac x (aYv 

w 

\— rJac x .(a)*M; y 



: v,w E ker(a x ) } C T c (T*(X) x T*(X)) 



is symplectomorphic to the symplectic normal space to S at £, Xs,^ = 
(T^E)" 1 /T^E. We shall denote by fi can the canonical symplectic structure of 
the cotangent bundles T*(X) and T*{X x X) = T*(X) x T*(X) at (x,ra x ) 
and at (, respectively. In our coordinate chart, this is the canonical sym- 
plectic structure on (M 2n+1 x R 2n+1 ) and (R 2n+1 x R 2n+1 ) x (R 2n+1 x R 2n+1 ). 
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We have, for v, w, v', w' G ker(a x ) horizontal lifts of £, r], rj' G T p M: 
( ( 



a 



V 





/ 




\ 


rJac x (o;)'f 




rJac x (a:)*i; ' 




w 




w' 




—7\Jac x («)*«; J 


V 


-rJac x (a)*i{// 


J 



v 

rJac x (a)'f 
= rn p (Z,?)-rn p (ri,ri'). 



rJac x (a:)V 



a 



w 

rJac x («)*«; 



Thus, there is a natural symplectic isomorphism 

7V S)C = (T P M, 2rQ p ) © (T P M, -2rfi p ), 



rJ&c x (ayw' 
(27) 

(28) 



span 



: a x (v) = ct(w) = 



which extends to an isomorphism of symplectic vector bundles with the ap- 
propriate symplectic structures. Let Jm be the complex structure of M. 
Then, when endowed with the compatible complex structure (Jm, — Jm), 
is a unitary (hence Riemannian) vector bundle. 

Let us next consider the vector subspace V£ C T^(T*(X x X)) \ {0} given 

by: 

/ v \ 

r Jac x (a:)i> 
w 

\— rJ&c x (a)w J 

Then is also naturally symplectomorphic to (T p M, —2rQ p ) © (T p M, 2rQ p ), 
p = ir(x), and thus becomes a unitary vector space with the complex struc- 
ture (—J p , J P ); it contains Tf(E) as a Lagrangian subspace. Clearly, it natu- 
rally extends to a vector subbundle of V C T(T*(X) x T*(X)). 

Next, let Kg : U — > IR 2ri+1 represent the vector field generating of the 
S^-action, and let us set k = k$(x). Let us then introduce the vector subspace 
W c C T ( (T*(X) x T*(X)) \ {0} given by 



= span 



I K \ 




/ 


o \ 


( ° ^ 


> 


rJa.c x (a)n 


ra 























K 


> 


V o J 


w 


V 


-ra J 


\—rJa,c x (a)n) 





Since k 1 ■ a x = a x ( | ) = 1, by mapping this (intrisically defined) basis to 
the real basis (e±, ie±, e 2 , ie 2 ) of C 2 we see that is naturally symplecto- 
morphic to (C 2 , tujq), where uq = ^ ^2 i=1 dziAdzi is the standard symplectic 
structure. We shall then consider as the fibre of a unitary vector bundle 
If on E. The proof of following is left to the reader: 
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Lemma 19. We have the symplectic direct sum decomposition 



T c (T*(X) x T*(X)) = 7V S)C © V c © W c . 

We may then take the direct sum of the unitary structures on each sum- 
mand to make T (T*(X) x T*(X))| E into a unitary vector bundle over S. 
We next consider the closed isotropic cone 

£' = {(x, ra' x , x, -ra' x ) : x G X' , r > 0} C T*(X') x T*(X') \ {0}, 

where a' = j*(cx). Let us fix (' = (x,ra' x ,x, —ra' x ) G £', and choose a 
coordinate patch [/' C X' containing x. Thus T*(C/') = £/' x R 2n+1 ~9, and 
a' is represented locally near i by a smooth function U' — > lR 2n+1_£ '. Let 
JaclfaO be its Jacobian matrix at x. Then in local coordinates 



T C ,(£') 



r Jac^.(a; / )f 



> © span 



/ \ 

«x 



V -< / 



\ -rJa<(a> / 

The symplectic annihilator is then T^E')- 1 = i?^/ + T^(£'), where 



R 



/ w \ 

rJa,c x (a'Yv 
w 

\ —r3ac f x (a'Yw J 

7 \ 



V ~9te J 



) span 



v,w G ker(a4 



(29) 



Here, for every (eg, denotes the linear functional on T X (X') given by 
the Riemannian scalar product with £"(#), where is the vector field on X' 
generated by £. Because of the degeneracy of the restricted form, the two 
summands for T^/ (£')-*- have a ^-dimensional intersection: 



Lemma 20. 



r c nr ( ,(E') 



/ £ tt (^) \ 

rJac>')^) 
\ -rJac>')^) J 
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Proof. Notice to begin with that the action of G on X is horizontal on 
X', that is, &(x) G ker(o4) if f G and x G X'. Let fi' = where 
t : M' <^-> M is the inclusion. Then Q' is a degenerate closed 2-form, whose 
kernel at any p G M' is the tangent space g-pC T P (M) to the orbit through 
p. On the other hand, it'* (Q') = da', where n' : X' — > M' is the projection. 
Since the space of all £"(x)'s is the horizontal lift of g • p at x, we deduce 
that in our local coordinates we have ^(x) G ker (Jac^(a') — Jac^(a')') for 
all (eg. On the upshot, ker(a4) Piker (Jac^(a') — Jacket;')*) is precisely the 
space of all £"(a;)'s, £ G g. Suppose now that we have an equality: 



/ 

rJ&c' x (a')v' 



X \ 

v' 



\ ( 

+ b 



\ —rJ&c' x (a')v' J 



\ 

a 'x 





/ 



\ 



rJauc'Ja'Yv 



x \ 
W 



\ —ria,c' x (a') t w J 



( \ 


V ~9te J 



V -< J 

with v, w G ker(a' ) and ( G g. Then v' = v = w G ker(a4) and so: 

r (Jacket') — Jac^.(a')')v + ba' x — g^ ;X = 0. 

The left hand side is a cotangent vector to X' at x. By pairing this first with 
the generator at x of the S^-action on X', and then with the ^(x)'s, rj G g, 
we obtain 6 = and £ = 0. Thus, t> = for some £ G g. 

Let H(M'/Mq) C TM' be the Riemannian orthocomplement of the ver- 
tical tangent bundle of Pm> '■ M' — > M . Then H(M' /M ) is a connec- 
tion for the principal G-bundle Pm 1 - When M' is endowed with the 2-form 
Q f = = p* M ,(Q ), H{M'/M ) is a symplectic vector subbundle of TM', 
symplectomorphic to the pull-back p* M , (TM ). Let £Ih(m' /m ) be its sym- 
plectic structure. Now in (|2~9j) may be decomposed as 

/ « \ 



rJ&c'Ja'Yv 



X \ 
W 



v, w G ker(o4), <i x 7r(u), d x 7r(w) G i7 p 



-r Jac^(a')*w / 



rJac^a')*^ 
\ rJac4(a')*£ tt (*) 7 
rJac^. (a'Y^(x) 

\ -rJac'V)*^) ) 



\ 



(eg 



£eg 



span 



/ \ 



V ~9te J 



(30) 
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By Lemma EDI the first two summands add up symplectomorphically to 
the symplectic normal bundle to £' in T*(X') x T*(X'). Thus, 

Lemma 21. Let N^, be the symplectic normal bundle of £' C T*(X') x 



T*(X'). Let C = (x,ra' x ,,x,-ra' x ,), p = n(x) e M,p = q(p) e M . Then 
we have a natural isomorphism of symplectic vector spaces: 

N v,C = (H P (M'/M ), 2r Q H{M , /Molp ) © (H p (M'/M ), -2r n H (M>/M ), P ) © . 

(Tp(M ) , rfi ,p) © (T ¥ (M ) , -rfi Q)F ) © 0C , P - 
T/ws naturally extends to an isomorphism of unitary vector bundles over £'. 
Here, 

/ e s w \ / o \ 

rJ&c' x (a')$(x) 



span 



9t,x 




On T P (M) D Tp(M'), we have in obvious notation g% = fi p (-, J p £}{p)). Thus, 
Qc,p may be naturally identified with the complexified Lie algebra of G, that 
is, the Lie algebra of the complexified group G. It is endowed with the unitary 
structure induced by its infinitesimal action on M at p, which makes it into 
a complex subspace of T P (M): gc tP = q c ■ p = T P {G ■ p). Let n' e denote the 
generator of the S^-action on X', set ) and let us now introduce 

the vector bundles over £' by setting, in local coordinates, 

/ v \ 

rJa.c x (a')v 



V> 



w 



:v,wE ker(a4), d x 7r(v),d p 7r(w) E H p 



\— rJ&c x (a')w J 
( 



V" 



W' c , 



span 



rJ&c' x (a')K 



x 

\ -rJao, J 

\ ( \ 

ra! 



/0\ 

9a 



w 





\-ra'J 



( \ 






k' 



\ 



\—rJ&c! x (a')K' J 



' 

V o / V o ) 

Just as before, these are naturally unitary vector bundles, and we have: 

Lemma 22. There is a symplectic direct sum decomposition 

T(T*(X) x T*(X))\v = ®V® V" © W . 

By taking the direct sum of the unitary structure of each summand, this 
makes T(T*(X) x T*(Jf))| s , into a unitary vector bundle. 
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6 The symbol of (j x j)* (fi x ). 



If W is a manifold, the symbol of a Fourier- Hermite distribution u G J k (W, S) 
associated to a closed isotropic cone S C T*W \ {0} is a smooth section of 
the symplectic spinor bundle, Spin(S), homogeneous of degree k with respect 
to the conic structure of H. The space of homogeneous sections of degree k 
of Spin(H) will be denoted by S k (E). 

In particular, the symbol of the Szego kernel fix G J X / 2 (A x A, E) is 
an element of S^E). The spinor bundle of E C T*(I x I) C {0} is 

1 /2 

Spin(S) = f\ £ Cg> S(iVs), where A s is the symplectic normal bundle of S. 
Now £ is obviously diffeomorphic to S Q in (}2~T]) . and by the symplecticity of 
the latter is a symplectic submanifold of T*X. 

Definition 1. By the above, £ carries a built-in symplectic structure ho- 

1 /2 

mogeneous of degree one. We shall denote by vol s the nowhere vanishing 
half-form of degree 1/2 obtained from the latter by the appropriate homog- 
enization with respect to the r coordinate. 

On the other hand, in view of (|2*Hj) . the symplectic normal bundle of E is 
(after an appropriate rescaling) naturally isomorphic to q*(TM) @q*{TM)~ . 
Thus, given the complex structure Jm of the base manifold M, by Corollary 
[HI the bundle S(N^) G End^ (S(q*(TM)) has a built-in nowhere vanishing 
section oj M - 

After |BGj . the symbol of the Szego kernel is the tensor product 



As an intermediate step towards computing the symbol of fbnx ; we shall 
now compute the symbol of the restriction (j x j)* I fix ] . 



If x G X' C X and rj G T*X is a cotangent vector to X at i, we shall 
use the notation rj' = (d x j)* (rj) G T*(X') for the restriction of rj to X' . The 
relevant canonical relation in T*(X' x X') x T*(X x X) is thus 

t jxj = {(( x i,Vi, x 2:V2)i x 2 ,r} 2 )) : Xi G X', rji G T*.(X)} , 

and the fibre product F JXJ of £ and IT^ maps diffeomorphically to £' under 
the projection p Fjxj : -> r jXj oS = £'. Let g : T*(X'x A') xT*(Ix A) -> 
T*(Ax A) be the projection onto the second factor. Fix ( = (x, ra, x, —ra) G 
£; the differential of q at ((', (), 



q c =: d {c , x) q : T f , (T*(A' x A')) x T c (T*(A x A)) ^ T c (T*(A x A)) , 




(31) 
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is simply projection onto the second factor. Then we have (in local coordi- 
nates) 



9C ( T (C',C)( r .w)) 



: v,w G T x (X'),(f),r) G 



p2n+l 



Given that X' = ($ o 7r) _1 (0), its tangent space T^X') is defined by the 
vanishing of all the differentials d x H^, £ G g, where =< £, $ > is 
the Hamiltonian function associated to £. Its symplectic annihilator in 
T C (T*(X) x T*(X)) is then 



9C ( T (C',C)( r w))" 



/ o \ 

d x H^ 


\d x H v ) 



It follows that {7oc =: ^ (^V.O^jxj)) ^ ^c(^) = i^i' while 
#iC = : ?C ( T (C',C)( r jx J )) ± n r f (S) ± = span 





( o ^ 






d x H^ 





J 




\-d x H $ J 





Thus, in view of the equality holding in local coordinates on U C X (see 
Claim E3} 

dflf = 6(X c )fi = ^ (Jac(a) - Jac(a) i )X 5 (£ G g), 

the vector space U K = U c C X SjC = (T P M, 2rfi p ) © (T P M, -2rfi p ) may be 
identified with the isotropic subspace 



C/ p ^{(X e (p),X£(p)):£G S } 



(32) 



where X^ us the vector field generated by £ G 0. As above, we shall make 
(T P M, 2rfl p ) © (T P M, —2rQ p ) into a unitary vector space, by endowing it with 
the compatible complex structure (J p , —J p ). Furthermore, let q m denote the 
trivial vector subbundle of TM\ M , on M' with fibre q generated by the 
infinitesimal action of q on M. Then g M is an oriented isotropic subbundle 
of TM\ M ,. We may write (|3~2j) as 



U = diag (g M ) C (T P M, 2rfi p ) © (T P M, -2rQ p 
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(33) 



Therefore, Corollary may be applied to the symplectic normal bundle Ny 
of U in (T p M, 2rQ p ) © (T p M, —2rQ p ). To this end, let us adopt the notation 
introduced in the discussion preceding (J30|) and in Corollary with L = g M , 
and recall after |GS2j that the symplectic and Riemannian annihilators of 
g M within TM\ M , are given by (g M ) X = g M © H(M'/M ) and (g A/ )° = 
Jm(5 m ) © H(M'/M ), respectively. 

Let us then define vector bundles g* f and gf 1 on M' by setting 0^(p) =: 
{pQ(p),-JQ(p)) : e G g} and gf (p) =: {( J p X^(p), J p X ( (p)) : £ G g} (p G 
M'), and then set (g M )c = sf © flf • 

By Corollary m), we conclude 

Corollary 9. the symplectic annihilator of U in iV s is 
Nu = (g A/ )c 

®(q*H(M'/M ), 2rQ H{M/A , n ) © {q*H{M'/M ), -2rQ H{M/M/) ). 

(34) 

Lemma 23. Ny is a metaplectic vector bundle. 
Proof. Let us consider the nested fibre diagrams 

£' JU X' ^ M' 
PE' I K' I I PM' (35) 

So X ^ M . 

The horizontal arrows are principal S^-bundles, and the vertical ones are 
principal G-bundles. Set q' = itx> ° p' '■ £' — ► M', g = ° Po : S — > M . It 
suffices to show that q'*{H(M' /Mq)) is a metaplectic vector bundle, and thus 
by commutativity that q^TMo) is metaplectic. This follows from Lemma IT7I 
applied to q . 

Equivalently, q* (H (M r / Mq))\ s , is the symplectic normal bundle of the 
oriented isotropic subbundle q* (q m ) | E , in the metaplectic vector bundle 
g* (TM ) | E , , and the fact that is is metaplectic then follows from Proposi- 
tion □ 

In view of Lemma and Corollary El we have 

S(N V ) = S (gg) © End HS (S(q'* H(M> /M ))) . 

Given that (g M )c is a complex vector subbundle of TM, the complex struc- 
ture J M of M singles out the section a a M t j M of S((g M ) c )). Since H(M'/M) = 
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p* M ,(TM ), the complex structure J of M singles out the section a Jo of 
End HS (S(q'*H(M'/M ))). Let <j Jm be as in and let §u : «S(iV E ) -> 
/\ 1 ^ 2 (t/) (g> S(Njj) be the morphism of vector bundles from Corollary 
Then, in view of Corollary iv), we have 

®(aj M ) = vo\-^\U) ® a M, jM ® a Jo , (36) 

where vol~ 1 ^ 2 (f/) is the —--form on U taking value one on oriented orthonor- 
mal frames of U. 

1/2 

Let now vol r _ x _ be the half-form on Tj x j associated to the morphism of 

metalinear manifolds J x J as in (JHJ), and let vol s be as in (|31jl. In the 
following definition, recall that the complex structure Jm maps the vector 
subbundle q m of TM\ M , isometrically onto the vector subbundle q m . 

Definition 2. Let e be the half-form on g M taking value one on oriented 
orthonormal frames. Given the exact sequence 

_ T(S') — > T(E)| S , — > J M (g M ) — ► 0, 

1 /2 1 

let vol E , be the half-form on £ , homogeneous of degree ^, obtained by 
dividing vol s ' 2 by e. 

Lemma 24. Suppose x £ X and let ( = ( ) (thus, r = 1). Then 

the image at ( o/volj/ 2 ^vol^igivol^y 2 m /\ 1 ^ 2 (S') ( ^ under the line bundle 

isomorphism \1$ is 2 9 vol^- 



The factor 2 9 is related to the relative dimension of the embedding j x j, 
which is 2g. A similar factor of 2~ g will appear in the following section, due 
to the fact that the relative dimension of px' X Px' is — %9- The two factors 
will thus cancel out in the final result. 

Proof. We shall henceforth omit the base point (. Let us fix an oriented 
orthonormal (real) basis {vj}i<j<2n of (T p M,2rfl p ), where p = ir(x), of the 
following form. First, let us choose an oriented orthonormal basis {vj} of 
H P (M'/M ) for 1 < j < 2{n - g). Next, let us set v 2 ( n -g)+j = £j(p), 1 < 
j — 9, where {^{p)} is any orthonormal frame of Q M (p) (the vectors £j G 
q depend on p), and finally let us set v 2ri -g+j = Jm,p^\{p)i 1 < j < 9- 
In particular, {vj}i<j<2n-g is an oriented orthonormal basis of T p M'. To 
simplify our expressions, we shall in the following denote by ~v the sequence 

7 = 2 Vl --7 = 2 V2(n - 9) 
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by £" the sequence 



and by J^" the sequence 



1 1 

With our usual slight abuse of language, we shall identify £' with the subset 
of E lying over diag(X'). As in section 5, we shall denote by k the genera- 
tor of the S^-action on X (expressed in local coordinates). As an oriented 
orthonormal basis of T^(S'), we shall then take 



( V \ 

JacJaW 



-•x \ 
V 



\-Jac :c (a)v / 



/V2 \ 



Jac T 

\-Jac :r (a)K/\/2 / 



/ o \ 



\-a x /V2 ) 



(37) 

Let us set <£>jx = | [Jac x (a)* — Jac x (a)] JX. We may extend By to a basis 
B r = By U B N of T c (r JXJ ) by letting 



B 



N 



( o \ / 







v \ 

JaCa;(a;)*v 



\-Jac a; (a)*v / 



/ e \ 

Jac x (a)*^ 
? 

V-Jac^a)*^ / 

Jac x (a)^ tt 

\ Jacket)? / 
/ \ / \ / K /s/2 \ 



( v W v \ 

Jac^a^'v JaCa;(a;)v 
—v —v 
\Jac :c (a)*v / \Jac :r (a)v / 



Jac x (a)*? 



9 





Vo y 








Jac T 



-K 



(38) 

Performing appropriate column operations, we obtain a basis whose first 
An + 2 vectors lie in T^*(X x X), while the remaining ones project down to 
an oriented orthonormal basis in T( XjX )(X' x X'). The statement follows from 
this and (jHJ). 
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This holds at any point where r — 1. Taking into account the appropriate 
homogeneity, on the upshot we have proved: 

Proposition 3. The symbol of (j x j)* ^ s ^ e sec ^ on °f Spin(S') = 

given by 

((x,ra' x ,x,-ra' x ) G £')■ 



7 The symbol of U x \x - 

We shall now examine the symbol of Ux\x , and relate it to the symbol of 
the Szego kernel of X , a(fl Xo ) G S 1/2 (£ )- 

Theorem 2. The symbol of Yix\x , denoted a(tlx\x ) £ S( 1+9 '' 2 (E ), is given 
by 

a(U x \x ){x,ra 0x ,x, -ra j) = W 2 V; ff (x) or(fl Xo )(x, ra 0£ , x, -ra £) 

(x G X Q , r > 0). 



Proof. We shall now be more sketchy, since the proof is similar to that of 
Proposition El By homogeneity, it suffices to prove the Theorem for r = 1. 

Let as above px> '■ X' —>■ X be the projection. The relevant canonical 
relation is now T PxlXPx , in (fT§j) . and it is naturally diffeomorphic to the 
horizontal cotangent bundle of px> x Px' ■ The fibre product F Px , XPx , of E' 
and T Px , XPx , is naturally diffeomorphic to The projection Pf Vx , XVx , '■ 
F Px , X p x , — S' — > T PxlXPx/ o E' = E is now the quotient map under the 
G-action. 

Let q' : T*(X x X ) x T*(X' x X') -> T*(X' x X') be the projection onto 
the second factor. Fix for now £' = (x, ra^,, x, — ra' x ) G the differential of 
<?' at (Co,C), 

q' c , =: d (Co)f0 g : T Co (T*(X x X )) x T ( , (T*(X' x X')) -> If, (T*(X' x X')) , 

is simply projection onto the second factor. To ease the exposition, we shall 
now assume that the local coordinates on the open neighbourhood U C X' of 
x have been so chosen that the vector fields £} are constant (since the action 
of G along <£> _1 (0) is free, this may certainly be done). 
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Lemma 25. In this system of local coordinates, we have 

J aCy (a')^(y) = Jac^ (</) = 0, 
for anyyeU,£e g. 

Proof. The two vanishings are equivalent, since the vector fields (65, 
span the kernel of da'. Since in addition the £"'s are horizontal on X', the 
asserted vanishings follow by differentiating the equality a'(£") = 0. 



In these local coordinates, 



?C' ( T (Co,C')( r jxj)) 



M 

w 



: v,w,(f>,r) G M 2n 9+1 such that 



0(£«(x))=7^»(x)) = O,V£e }. 
The symplectic annihilator in Tp(T*(X) x T*(X)) is then 



?C ( T (C',o( r jxj))" 





77" (x) 

V / 



We shall now denote by V , V\ and V the analogues in the present setting of 
the vector bundles Uq, U\ and U introduced in section 3. In view of Lemma 
we now have 





V / 



: ( G g > and V\ 














t 1 


J 


77" (x) 






\ J 





Hence, in the terminology of Corollary iii) and recalling Corollary El we 
obtain for the quotient bundle V = Vl/Vq C jVjy = N v : 



( i\x) \ 


V y 



£e V =gf(p)civ ; 



Again by Corollary [7[ the symplectic normal bundle Ny of V in iV^y is thus 
AV (q*H(M'/M ),2rn HiM/M/) ) © (q*H(M'/M ), -2rQ H{M /M'))- 
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This is clearly simplectically isomorphic to the normal bundle of S in T*(X x 
Xq) (cfr Proposition 6.4 of |BGj ). By Corollary 0v), the image of <J s M j®aj a 
under the vector bundle morphism $y : S{N^) — > f\ (V) <g> S(N V ) is 

where voly 1 ^ 2 is the — |-form on the oriented isotropic subbundle V C T(T*(X'x 
X')) taking value one on oriented ortho normal basis of V. 

The following Lemma is proved with an argument similar to that used 
for Lemma |H1 the main difference being we need to include an oriented or- 
tho normal basis for Vo^'. 

Lemma 26. Suppose and let (' = (x, a' x , x, —oc' x ) (thus, r = 1). Then 

the image at (' of vol^/ 2 xp ^ ® vol^, <8> vol y ^ 2 in /\ 1 ^ 2 (S )( <8> det(V^,) 

under the line bundle isomorphism \1$ is 2~ g vol^/ 2 ^ <S> v oly ,/ , where voly 
is the volume form on Vq taking value one on oriented orthonormal frames. 

Clearly, voly o , is the volume form on the vertical tangent bundle of the 
principal G-bundle £' — > S associated to the orientation and the restricted 
metric. The statement of Theorem [2] then follows by fibrewise integration 
and homogenization. 



8 The Asymptotic Expansion 



The function Ej=i s j ' \ x ') ® s j ' (z/0 appearing in (|TH|) is obviously well- 
defined on Xq. We shall now argue that it admits an asymptotic expansion 
as in the statement of Theorem ^ 

To this end, let us introduce the following auxiliary Fourier-Hermite dis- 
tribution on Xo x Xo: 

Px\x =: {V eS ®V cS )- l fl xlXo 

Thus Px\x £ J( 1+9 >' 2 (X x X ,E ), and the associated operator Px\x '■ 
Vy 2 (X ) — > T>[, 2 (X ) is an ^-invariant elliptic operator satisfying Ilx ° 
Px\x = Px\x - One can now follow the arguments in |STZj . Lemma 4.2 and 
4.3 and [0] to conclude that Px\x , and thus IIx|Xo> is an elliptic Toeplitz 
operator possessing a semiclassical symbol, and that this implies the asserted 
asymptotic expansion. 
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